[cxx-interop] Add support for benchmarks with C++ interop. by zoecarver · Pull Request #32721 · swiftlang/swift

zoecarver · 2020-07-06T16:38:33Z

Adds support for benchmarks that use C++ modules.
Adds "CreateObjects" benchmark that creates C++ objects.

Based on #32715.

MForster

This seems to contain a lot of unrelated changes...

benchmark/cxx-source/CreateObjects.swift

benchmark/cmake/modules/AddSwiftBenchmarkSuite.cmake

zoecarver · 2020-07-07T16:20:53Z

This seems to contain a lot of unrelated changes...

Yes, this PR is based on #32715 so all the changes from that PR are here too (just so it compiles).

MForster

(after the base pull request has been merged)

zoecarver · 2020-07-07T17:30:56Z

@MForster sounds good. Thanks!

gribozavr

@zoecarver This PR is adding benchmark/.swiftpm/xcode/package.xcworkspace/xcuserdata/zoe.xcuserdatad/UserInterfaceState.xcuserstate, I think that's not intentional.

benchmark/utils/CxxTests/cxx-types.h

gribozavr · 2020-07-08T15:36:57Z

lib/IRGen/IRGenSIL.cpp

Is there a separate PR for these changes to the compiler?

No, there's not. I'll make one.

Here it is: #32815.

zoecarver · 2020-07-08T16:45:11Z

This PR is adding benchmark/.swiftpm/xcode/package.xcworkspace/xcuserdata/zoe.xcuserdatad/UserInterfaceState.xcuserstate, I think that's not intentional.

Oops, you're right. Xcode adds that when I open the benchmarks/ directory. Forgot to remove it last time I committed.

benchmark/utils/CxxTests/CreateObjects.h

gribozavr · 2020-07-17T13:50:06Z

LGTM! Please rebase and merge after #32815 goes in.

zoecarver · 2020-07-23T22:51:42Z

@swift-ci please test and merge

zoecarver · 2020-07-23T22:52:04Z

@swift-ci please smoke test and merge

zoecarver · 2020-07-23T22:52:25Z

@swift-ci please smoke test and merge

zoecarver · 2020-07-24T18:47:24Z

@swift-ci please test Linux platform.

zoecarver · 2020-07-24T18:48:25Z

@swift-ci please test Linux platform

zoecarver · 2020-07-24T19:39:08Z

@swift-ci please benchmark

zoecarver · 2020-07-24T19:39:19Z

@swift-ci please test windows platform.

zoecarver · 2020-07-24T19:39:32Z

@swift-ci please benchmark

zoecarver · 2020-07-24T19:39:55Z

@swift-ci please benchmark.

zoecarver · 2020-07-24T19:40:15Z

@swift-ci please benchmark

zoecarver · 2020-07-24T19:51:10Z

@swift-ci please benchmark

swift-ci · 2020-07-24T20:36:55Z

Build failed
Swift Test Linux Platform
Git Sha - 2e389d641f88977b77cf14ea612fa7d479499eb0

swift-ci · 2020-07-24T20:50:54Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
NSStringConversion.MutableCopy.Rebridge.Long	996	1072	+7.6%	0.93x (?)

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	1	1	1	—

Code size: -O

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
FlattenListFlatMap	4123	6572	+59.4%	0.63x (?)
ObjectiveCBridgeStubFromNSDate	5980	7150	+19.6%	0.84x (?)
CharacterLiteralsLarge	100	111	+11.0%	0.90x (?)
DataSetCountMedium	320	350	+9.4%	0.91x (?)

Improvement	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubDateAccess	285	257	-9.8%	1.11x
ObjectiveCBridgeStubToNSDateRef	3900	3520	-9.7%	1.11x (?)

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	2	2	2	—

Code size: -Osize

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubFromNSDate	5840	6860	+17.5%	0.85x (?)
StringBuilderWithLongSubstring	2910	3200	+10.0%	0.91x (?)

Improvement	OLD	NEW	DELTA	RATIO
CStringLongAscii	321	288	-10.3%	1.11x (?)
Hanoi	13860	12770	-7.9%	1.09x (?)

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	190	203	195	—

Code size: -swiftlibs

✅	Benchmark Check Report
⚠️⏱	`CreateObjects` execution took 1 μs. _{Increase the workload of CreateObjects to be more than 20 μs.}

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

zoecarver · 2020-09-09T03:53:11Z

@swift-ci please benchmark

zoecarver · 2020-09-09T03:53:18Z

@swift-ci please smoke test

swift-ci · 2020-09-09T04:23:30Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
ArrayPlusEqualSingleElementCollection	423	470	+11.1%	0.90x (?)
DataSubscriptMedium	37	41	+10.8%	0.90x (?)

Improvement	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubDateAccess	152	130	-14.5%	1.17x

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	11	11	11	—

Code size: -O

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
Dictionary4	157	197	+25.5%	0.80x (?)
StringWalk	1640	1800	+9.8%	0.91x (?)

Improvement	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubDateAccess	152	130	-14.5%	1.17x
CharacterLiteralsLarge	71	63	-11.3%	1.13x (?)
CharacterLiteralsSmall	228	211	-7.5%	1.08x (?)
ArrayLiteral2	116	108	-6.9%	1.07x (?)

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	11	11	11	—

Code size: -Osize

Performance: -Onone

Added	MIN	MAX	MEAN	MAX_RSS
CreateObjects	1391	1459	1429	—

Code size: -swiftlibs

✅	Benchmark Check Report
⚠️⏱	`CreateObjects` execution took 11 μs. _{Increase the workload of CreateObjects to be more than 20 μs.}

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: 6-Core Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

eeckstein · 2020-09-09T09:17:53Z

benchmark/cxx-source/CreateObjects.swift

I assume this benchmark is just there to validate that C++ support is compiling fine.
I think it makes sense to disable this benchmark from the regular run, because it's not doing anything interesting.
Can you add the ".skip" benchmark tag?

Actually, I'd like to test the performance of creating C++ objects. For example, I expect that this will be much slower after #30630 (when we actually have to call the constructor instead of just allocating an empty object).

I see. Makes sense

zoecarver · 2020-09-10T19:42:41Z

@swift-ci please smoke test Linux platform.

zoecarver · 2020-09-10T19:43:09Z

@swift-ci please smoke test Linux platform

zoecarver · 2020-09-10T23:44:22Z

@swift-ci please smoke test Linux platform

zoecarver · 2020-09-10T23:44:40Z

@swift-ci please smoke test Linux platform

zoecarver · 2020-09-11T17:21:23Z

The build failure appears to be unrelated.

zoecarver · 2020-09-11T17:21:32Z

@swift-ci please smoke test Linux platform.

shahmishal · 2020-10-01T06:53:59Z

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

How to change the base branch: (Link)
More detail about the branch update: (Link)

* Adds support for benchmarks that use C++ modules. * Adds "CreateObjects" benchmark that creates C++ objects.

zoecarver · 2020-10-07T02:39:47Z

@swift-ci please smoke test Linux platform.

zoecarver · 2020-10-07T06:23:01Z

Failed because of an unrelated issue.

@swift-ci please smoke test Linux platform.

MaxDesiatov · 2020-10-14T17:06:52Z

swiftlang/swift-corelibs-foundation#2895
@swift-ci please smoke test

zoecarver · 2020-11-20T07:40:48Z

@swift-ci please test.

zoecarver · 2020-11-20T07:41:03Z

@swift-ci please test.

zoecarver added the c++ interop Feature: Interoperability with C++ label Jul 6, 2020

zoecarver requested review from MForster and gribozavr July 6, 2020 16:38

MForster suggested changes Jul 7, 2020

View reviewed changes

benchmark/cxx-source/CreateObjects.swift Outdated Show resolved Hide resolved

benchmark/cmake/modules/AddSwiftBenchmarkSuite.cmake Outdated Show resolved Hide resolved

zoecarver force-pushed the cxx/benchmark/create-objects branch from 4e31904 to 0a88a45 Compare July 7, 2020 16:49

MForster approved these changes Jul 7, 2020

View reviewed changes

gribozavr reviewed Jul 8, 2020

View reviewed changes

zoecarver force-pushed the cxx/benchmark/create-objects branch from 0a88a45 to 059c6b0 Compare July 8, 2020 16:57

zoecarver mentioned this pull request Jul 10, 2020

[cxx-interop] Temporarily disable emitting debug info for C++ types. #32815

Merged

zoecarver force-pushed the cxx/benchmark/create-objects branch from 059c6b0 to df25b96 Compare July 10, 2020 16:17

gribozavr reviewed Jul 17, 2020

View reviewed changes

benchmark/utils/CxxTests/CreateObjects.h Outdated Show resolved Hide resolved

gribozavr reviewed Jul 17, 2020

View reviewed changes

benchmark/utils/CxxTests/CreateObjects.h Outdated Show resolved Hide resolved

gribozavr approved these changes Jul 17, 2020

View reviewed changes

zoecarver force-pushed the cxx/benchmark/create-objects branch 2 times, most recently from d488500 to 2e389d6 Compare July 23, 2020 22:51

zoecarver force-pushed the cxx/benchmark/create-objects branch 2 times, most recently from 28ee166 to f79f5be Compare September 9, 2020 03:53

eeckstein reviewed Sep 9, 2020

View reviewed changes

zoecarver force-pushed the cxx/benchmark/create-objects branch from f79f5be to c1235c2 Compare September 10, 2020 19:41

zoecarver force-pushed the cxx/benchmark/create-objects branch from c1235c2 to 19b875a Compare September 10, 2020 23:44

zoecarver changed the base branch from master to main October 2, 2020 04:03

[cxx-interop] Add support for benchmarks with C++ interop.

01b12cf

* Adds support for benchmarks that use C++ modules. * Adds "CreateObjects" benchmark that creates C++ objects.

zoecarver force-pushed the cxx/benchmark/create-objects branch from 19b875a to 01b12cf Compare October 7, 2020 02:39

This was referenced Oct 8, 2020

[cxx-interop] Fix two headers to be valid in C++ mode. swiftlang/swift-corelibs-foundation#2895

Merged

[cxx-interop] Cast "data" to the correct type in _swift_dispatch_data_apply. #34266

Merged

zoecarver mentioned this pull request Oct 20, 2020

Add support for calling C++ constructors #30630

Merged

zoecarver merged commit aa04872 into swiftlang:main Nov 20, 2020

Conversation

zoecarver commented Jul 6, 2020

Uh oh!

MForster left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zoecarver commented Jul 7, 2020

Uh oh!

MForster left a comment

Choose a reason for hiding this comment

Uh oh!

zoecarver commented Jul 7, 2020

Uh oh!

gribozavr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gribozavr Jul 8, 2020

Choose a reason for hiding this comment

Uh oh!

zoecarver Jul 8, 2020

Choose a reason for hiding this comment

Uh oh!

zoecarver Jul 10, 2020

Choose a reason for hiding this comment

Uh oh!

zoecarver commented Jul 8, 2020

Uh oh!

Uh oh!

Uh oh!

gribozavr commented Jul 17, 2020

Uh oh!

zoecarver commented Jul 23, 2020

Uh oh!

zoecarver commented Jul 23, 2020

Uh oh!

zoecarver commented Jul 23, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

zoecarver commented Jul 24, 2020

Uh oh!

swift-ci commented Jul 24, 2020

Uh oh!

swift-ci commented Jul 24, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

zoecarver commented Sep 9, 2020

Uh oh!

zoecarver commented Sep 9, 2020

Uh oh!

swift-ci commented Sep 9, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

eeckstein Sep 9, 2020

Choose a reason for hiding this comment